Diffusion-based generative models, as first introduced by Sohl-Dickstein et al. (2015), have shown

The rapid scaling of language models is motivating research using low-bitwidth quantization. In this work, we propose a novel binarization technique for Transformers applied to machine translation (BMT), the first of its kind.

We present ComSL, a speech-language model built atop a composite architecture of public pretrained speech-only and language-only models and optimized data-efficiently for spoken language tasks.

Neural Information Processing Systems http://nips.cc/

To explain these observations, we visualize how label smoothing changes therepresentations learned bythepenultimate layerofthenetwork. We show that label smoothing encourages the representations of training examples from thesame class togroup intight clusters. This results inloss ofinformation inthe logits about resemblances between instances ofdifferent classes, which isnecessary for distillation, but does not hurt generalization or calibration of the model'spredictions.

Neural Machine Translation (NMT) has achieved remarkable progress with the quick evolvement of model structures. In this paper, we propose the concept of layer-wise coordination for NMT, which explicitly coordinates the learning of hidden representations of the encoder and decoder together layer by layer,gradually from lowleveltohigh level.